Papers by Roelien C. Timmer
A Position Paper on the Automatic Generation of Machine Learning Leaderboards (2025.emnlp-main)
Copied to clipboard
| Challenge: | Automated leaderboard generation is a tool for comparing prior work with a tabular overview of experimental results. |
| Approach: | They propose an automatic leaderboard generation framework to standardise how the task is defined. |
| Outcome: | The proposed framework standardises how the ALG task is defined and proposes new directions . the proposed framework includes recommendations for datasets and metrics that promote fair evaluation . |
MetaLead: A Comprehensive Human-Curated Leaderboard Dataset for Transparent Reporting of Machine Learning Experiments (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing leaderboards capture only the best results from each paper and have limited metadata. |
| Approach: | They propose to create a fully human-annotated ML Leaderboard dataset that captures all experimental results and contains extra metadata. |
| Outcome: | The MetaLead dataset captures all experimental results and contains extra metadata for cross-domain evaluation. |